CPU Torrent – CPU Cycle Offloading to Reduce User Wait Time and Provider Resource Requirements

Developers of novel scientific computing systems are often eager to make their algorithms and databases available for community use, but their own computational resources may be inadequate to fulfill external user demand – yet the system’s footprint is far too large for prospective user organizations to download and run locally. Some heavyweight systems have become part of designated “centers” providing remote access to supercomputers and/or clusters supported by substantial government funding; others use virtual supercomputers dispersed across grids formed by massive numbers of volunteer Internet-connected computers. But public funds are limited and not all systems are amenable to huge-scale divisibility into independent computation units. We have identified a class of scientific computing systems where “utility” sub-jobs can be offloaded to any of several alternative providers thereby freeing up local cycles for the main proprietary jobs, implemented a proofof-concept framework enabling such deployments, and analyzed its expected throughput and response-time impact on a real-world bioinformatics system (Columbia’s PredictProtein) whose present users endure long wait queues.